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DETAILED ACTION 



1 . Response to communications filed on February 20, 2004. 

2. Applicant's arguments with respect to claims 1 - 33 have been considered but are 
moot in view of the new ground(s) of rejection. 



3. Claims 11,14, 19 and 23 are objected to as being dependent upon a rejected base 
claim, but would be allowable if rewritten in independent form including all of the 
limitations of the base claim and any intervening claims. 

The followings is a statement of reasons for the indication of allowable subject 
matter: The prior art of record fails to teach and/or suggest "the step of examining the 
information stream further . . ., identifying a second set of documents that correspond to 
words found in the text; scoring the returned documents based on a plurality of criteria 
and ranking the documents based on their scores". 



4. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as 
set forth in section 102 of this title, if the differences between the subject matter sought to be 
patented and the prior art are such that the subject matter as a whole would have been obvious 
at the time the invention was made to a person having ordinary skill in the art to which said 
subject matter pertains. Patentability shall not be negatived by the manner in which the invention 



This application currently names joint inventors. In considering patentability of 



Claim Objections 



Claim Rejections - 35 USC § 103 



was made. 



the claims under 35 U.S.C. 103(a), the examiner presumes that the subject matter of 
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the various claims was commonly owned at the time any inventions covered therein 
were made absent any evidence to the contrary. Applicant is advised of the obligation 
under 37 CFR 1 .56 to point out the inventor and invention dates of each claim that was 
not commonly owned at the time a later invention was made in order for the examiner to 
consider the applicability of 35 U.S.C. 103(c) and potential 35 U.S.C. 102(e), (f) or (g) 
prior art under 35 U.S.C. 1 03(a). 

5. Claims 1, 2, 3, 4, 18, 24 and 33 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over U.S. Patent 5,500,920 issued to Julian M. Kupiec (hereinafter 
"Kupiec") in view U.S. Patent 5,625,767 issued to Brian Bartell et al (hereinafter 
"Bartell"). 

Regarding claims 1, 18, 24 and 33, Kupiec teaches a method for providing 
collateral information for inclusion with an information stream, comprising steps of: 

examining the information stream to recognize a presence of events that occur in 
the information stream (see column 14, lines 15-20; "Kupiec discloses the phone 
sequence as information stream"). 

Kupiec does not explicitly teach wherein said events are derived from the 
information stream based on one or more predetermined taxonomies; 

automatically generating database queries from derived events. 
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analyzing results of said database queries so as to rank and select said results to 
be inserted into the information stream as information that is collateral to said derived 
events. 

Bartell teaches said events are derived from the information stream based on 
one or more predetermined taxonomies (see column 2, lines 11-14); 

automatically generating database queries from derived events (see column 1 , 
lines 14-35). 

analyzing results of said database queries so as to rank and select said results to 
be inserted into the information stream as information that is collateral to said derived 
events (see column 2, lines 11 - 21). 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine teaching of Bartell with the teaching of Kupiec wherein 
classification documents under a predetermined set of classes or a predetermined 
hierarchical taxonomy create a pool of event where the information is examined. The 
motivation is that is that this pool can be analyzed and queried to extract desired 
information. 

Regarding claim 2, Kupiec teaches the step of analyzing comprises a step of 
ranking the database query results based on a plurality of criteria (see column 12, lines 
1 - 6 and column 17, lines 1 - 4). 
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Regarding claim 3, Kupiec teaches the plurality of criteria comprise a score 
derived from a free text search of the database using text that is automatically extracted 
from the information stream, on a number of named entities appearing in the text and in 
the database query results, and on a taxonomy path score, where the taxonomy path 
score represents an amount of relatedness between a taxonomy-related information 
element found in the text and a tree of the predetermined taxonomies (see column 16, 
lines 59 - 67 and column 17, lines 1 - 35). 

Regarding claim 4, Bartell teaches the database queries are automatically 
generated based on information corresponding to a list that identifies topics in text that 
is automatically extracted from the information stream, where the topics correspond to 
topic taxonomies of the predetermined taxonomies (see column 1, lines 14-35 and 
column 2, lines 11-14). 

6. Claims 5 - 10, 20 - 22, and 25 - 32 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Kupiec in view of Bartell and further in view of U.S. Patent 
5,835,667 issued to Howard D. Wactlar et al (hereinafter "Wactlar"). 

Regarding claim 5, Kupiec or Bartell does not explicitly teach extracting text. 
Wactlar teaches the step of examining comprises a step of automatically 
extracting text from the infornnation stream (see column 12, lines 52 - 58). 
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It would have been obvious to one of ordinary skill in the art at the tinne the 
invention was nnade to combine teaching of Wactlar with the teaching of Kupiec and 
Bartell wherein textual information are extracted from audio/video data. The motivation 
is that this data could be analyzed, searched and manipulated for desired segments. 

Regarding claim 6, Kupiec teaches segmenting the text into sentences (see 
column 24, lines 51 - 55); and 

operating on the sentences to identify topics that correspond to predetermined 
topic taxonomies, wherein the step of automatically generating database queries 
operates on identified topics (see column 16, lines 30 - 36). 

Regarding claim 7, Wactlar teaches the step of automatically extracting text from 
the information stream comprises a step of operating a voice recognition system (see 
column 6, lines 57 - 58) 

Regarding claim 8, Wactlar teaches the step of automatically extracting text from the 
information stream comprises a step of extracting closed caption text (see column 7, 
lines 58 - 62). 
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Regarding clainn 9, Kupiec teaches the step of automatically extracting text from 
the information stream comprises a step of operating a character recognition system 
(see column 24, lines 32 - 35). 

Regarding claim 10, Wactlar teaches the step of automatically extracting text from 
the information stream comprises a step of also generating text that is descriptive of a 
number of human faces that are present in an image conveyed by the information 
stream (see Figs. A-1 and A-2). 

Regarding claim 20, Wactlar teaches examining subsystem comprises at least one 
unit for automatically extracting text from the information stream, a unit for segmenting 
the text into sentences and at least one unit for operating on the sentences to identify 
topics that correspond to topic taxonomies of the predetermined taxonomies, wherein 
said query generation subsystem automatically generates database queries based at 
least in part on identified topics (see column 12, lines 57 - 58). 

Regarding claim 21 , Kupiec teaches text extracting unit comprises at least one of a 
voice recognition system, a system for extracting closed caption text, and a character 
recognition system (see column 24, lines 32 - 35). 
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Regarding claim 22, Wactlar teaches examining subsystem comprises a unit for 
generating text that is descriptive of a number of human faces that are present in an 
image conveyed by the information stream (see Figs.A-1 and A-2). 

Regarding claim 25, Wactlar teaches a step of inserting the collateral information 
into the audio/video stream in real time or substantially real time (see column 6, lines 36 



Regarding claim 26, Wactlar teaches examining includes a step of generating a 
speech transcript from at least the audio portion of the audio/visual stream, and wherein 
recognized events comprise speech topics (see column 6, lines 41 - 48). 

Regarding claim 27, Wactlar teaches the audio/video stream originates as a 
television broadcast signal (see column 1 , lines 25 - 33). 

Regarding claim 28, Wactlar teaches the audio/video stream originates at a meeting, 
and further comprising a step of presenting the collateral information to meeting 
participants in real time or substantially real time (see column 1, lines 14 - 18 and 30 - 



-38). 



33). 
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Regarding clainn 29, Wactlar teaches the step of presenting comprises a step of 
inserting the collateral information into the audio/video stream, and displaying the 
audio/video stream to the meeting participants (see Fig.4 step 42). 

Regarding claim 30, Wactlar a step of archiving at least the collateral information 
(see column 6, lines 46 - 50). 

Regarding claim 31 , Wactlar teaches the database queries are automatically 
generated based on information corresponding to identified topics extracted from the 
audio/video stream, where the topics correspond to topic taxonomies of the 
predetermined taxonomies (see column 11, lines 50 - 51). 

Regarding claim 32, Wactlar the step of examining includes steps of generating a 
speech transcript comprised of words from at least the audio portion of the audio/video 
stream; segmenting the words into sentences; and operating on the sentences to 
identify topics that correspond to topic taxonomies of the predetermined taxonomies, 
wherein the step of generating database queries operates on identified topics (see 
column 2, lines 44 - 53). 
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7. Claims 12, 13, 15, 16 and 17 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Wactlar in view of Bartell. 

Regarding claim 12, Wactlar teaches a method for providing collateral information 
for multiplexing with an information stream, comprising steps of: 

converting the information stream into text (see column 6, lines 46 - 50); 

extracting data from database search results that is relevant to the information 
stream (see column 15, lines 2 - 6); 

and multiplexing the data into the information stream for presentation at a destination 
of the information stream (see column 15, lines 20 - 67). 

Wactlar does not explicitly teach analyzing the text to identify information elements 
based on one or more predetermined taxonomies; automatically generating queries 
from the information elements for searching at least one database. 

Bartell teaches teach analyzing the text to identify information elements based on 
one or more predetermined taxonomies (see column 2, lines 11 - 21 ); 

automatically generating queries from the information elements for searching at least 
one database (see column 1, lines 14-35 and column 2, lines 11-14); 

It would have been obvious to one of ordinary skill in the art at the time the invention 
was made to combine teaching of Bartell with teaching of Wactlar wherein the queries 
are used in selecting desire audio/video stream. The motivation is that automatically 
generating these data base query saves time. 
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Regarding claim 13, Wactlar the step of extracting comprises a step of ranl<lng the 
extracted information based on a plurality of criteria, and where the step of multiplexing 
uses the ranked data (see column 10, lines 15 - 22). 

Regarding claim 15, Wactlar teaches the queries are generated based on 
information elements that correspond to a list of information elements identifying topics 
in the text being analyzed, where the topics correspond to topic taxonomies of the 
predetermined taxonomies (see column 6, line 67 and column 14, lines 46 - 57). 

Regarding claim 16, Wactlar teaches the step of analyzing the text comprises steps 
of segmenting the text into sentences and a step of operating on the sentences to 
identify topics that correspond to topic taxonomies of the predetermined taxonomies, 
and wherein the step of automatically generating queries operates on identified topics 
(see column 12, lines 57 - 58). 

Regarding claim 17, Wactlar teaches the step of analyzing the text comprises steps 
of at least segmenting the text into sentences, identifying names of entities within the 
text, and a step of operating on the sentences to identify topics that correspond to topic 
taxonomies of the predetermined taxonomies, and wherein the step of automatically 
generating queries operates on identified topics and ranks the database search results 
based at least on numbers of named entities found and on an amount of relatedness 
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between a taxonomy-related information element identified in the text and a tree of the 
predetermined taxonomies (see column 9, lines 41 - 56). 



8. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Fred I. Ehichioya whose telephone number is 703-305- 
8039. The examiner can normally be reached on M - F 8:00 AM to 4:30 PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, John E. Breene can be reached on 703-305-9790. The fax phone number 
for the organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 



Conclusion 



Fred I. Ehichioya 
Examiner 
Art Unit 2172 
March 3, 2004 




